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Abstract 

The  dynamic  programming  and  branch-and-bound  approaches  are 
combined  to  produce  a  hybrid  algorithm  for  separable  discrete 
mathematical   programs.     The  hybrid  algorithm  uses  linear  programming 
in  a  novel  way  to  compute  bounds  and  is  readily  extended  to  solve 
a  family  of  parametric  integer  programs  with  related  right-hand-sides. 
Computational  experience  is  reported  on  a  number  of  linear  and  nonlinear 
integer  programs. 
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1.  INTRODUCTION 

This  paper  presents  a  new  approach  to  the  solution  of  separable  discrete 
mathematical  programs.  The  approach  is  a  synthesis  of  dynamic  programming  (DP) 
and  branch-and-bound  (B&B).  Relaxations  and  fathoming  criteria,  which  are 
fundamental  to  branch-and-bound,  are  incorporated  within  the  separation  and 
initial  fathoming  provided  by  the  dynamic  programming  framework  in  order  to 
produce  a  hybrid  DP/B&B  algorithm. 

The  general  separable  discrete  mathematical  program  we  address  is: 

N 
f  (b)  =  max  I       r.(x.)  (1.1) 


l<i^ 

l<j^N 

where  S.  =  {0,  1,  ...,  K. }  with  K.  a  finite  positive  integer.  To  simplify  the 
motivation  and  exposition  we  shall  begin  by  making  a  non-negativity  assumption 
on  all  of  the  problem  data: 

b.>  0  Ui<M 

r(xj)>  0  l<jsN,  X.   £  S. 

a.j(Xj)>  0  l<i<M.  l<j<N,  X.   e  S... 

This  makes  (1.1)  a  "knapsack  type"  resource  allocation  problem  [1,5,  25,  34] 
which  can  be  given  the  following  interpretation.  The  amount  of  resource  i 
available  is  b^.  and  if  project  j  is  adopted  at  level  x.  then  a..(x.)  is  the 
amount  of  resource  i  consumed  and  r.(x.)  is  the  return.  The  non-negativity 
assumption  will  remain  in  force  until  Section  7.  We  may  further  assume,  without 
loss  of  generality,  that  r.(0)=0  and  a..(0)=0  for  all  i,j.  Notice  that  if  K.=l 

J  I  J  J 

for  all  j,  then  (1.1)  is  the  familiar  zero/one  integer  linear  program. 


subject  to 

N 

(Xj) 

s  b 

"j' 

'i 
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The  hybrid  DP/B&B  algorithm  has  grown  out  of  the  authors'  earlier  work 
on  a  DP  algorithm  for  knapsack  type  problems  [21]  and  the  observation  that 
bounding  tests  can  be  used  to  reduce  the  state  space  in  DP  [22]«  Thus,  ideas 
from  B&B  can  dramatically  enhance  the  computational  power  of  DP.  The  hybrid 
algorithm  may  be  viewed  as  a  DP  recursion  which  uses  bounding  tests  at  each 
stage  to  eliminate  (fathom)  some  of  the  states.  Alternatively,  it  may  be 
viewed  as  a  B&B  tree  search  which  uses  elimination  by  dominance,  as  well  as  by 
bound,  and  which  employs  the  ultimate  "breadth  first"  search  strategy.  The 
partitioning  of  the  problem  into  stages,  which  is  inherited  from  DP,  leads 
directly  to  a  new  way  of  using  linear  programming  to  compute  bounds.  This  is 
called  the  resource-space  tour  and  it  has  the  attractive  feature  that  each 
simplex  pivot  yields  a  bound  for  every   active  node  in  the  search  tree.  The  DP 
point  of  view  also  focuses  attention  on  the  optimal  return  function  ffjCb)  and 
leads  to  a  procedure  for  solving  a  family  of  parametric  integer  programs  with 
related  right-hand-sides. 

The  plan  of  the  paper  is  as  follows.  The  hybrid  approach  will  be  developed 
in  Section  2,  assuming  the  availability  of  an  algorithm  for  solving  some  relax- 
ation of  problem  (1.1)  and  of  a  heuristic  for  finding  feasible  solutions  of 
(1.1).  Specific  relaxations  and  heuristics  will  be  discussed  in  Sections  3  and 
4,  respectively.  The  resource-space  tour  technique  for  computing  bounds  is 
introduced  in  Section  3.  Section  5  contains  a  summary  of  computational  results. 
The  extension  of  the  hybrid  algorithm  to  solve  a  family  of  parametric  integer 
programs  is  done  in  Section  6.  In  Section  7,  the  modifications  required  for 
the  general  case  (positive  and  negative  data)  are  indicated.  Suggestions  for 
further  research  are  given  in  Section  8. 

Related  work  on  the  synthesis  of  branch-and-bound  with  dynamic  programming 
can  be  found  in  [  i,  4,  6,  7,  14,  15,  24,  31,  37], 
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2.      Development  of   the  Hybrid  Algorithm 


Consider  the   following  n-stage   subproblem  of   (1.1) 

n 
f   (b)    =  max  y        r.(x.)  (2.1) 

j=l     J      J 


subject   to 
n 


y      a. .(x.)    <  b. 
J=l 


l<i<M 


X.    e      S.  l^i^n 

J  J 


for  n  =   1,    ...,   N.      Let  X     denote  the   set   of   feasible  solutions   of   (2.1). 

n 

The  feasible  solution  x  e  X  is  said  to  be  dominated  by  the  feasible  solution 

n  

x'  e  X  if  we  have  both 
n 


I      a  (xO  <  I        a   (X  ) 
j=l  "-J   J    j=l  "-J   J 


l<i<M 


and 


n  n 

I        r.(xr)  >  I        r.(x.) 

j=l  J   J    j=l  J   J 


with  at  least  one  strict  inequality.   If  x  e  X  is  not  dominated  by  any 

other  element  of  X  ,  then  we  say  that  x  is  efficient  with  respect  to  X 
n  -^  n 

Q 

Let  X  denote  the  set  of  efficient  solutions  of  (2,1). 
n 

The  set  X^  of  all  efficient  solutions  of  the  complete  problem  (1.1) 
can  be  constructed  recursively  by  using  the  following  relationships: 


X£  c  x^  c  x:   =  S 


0 

1   "1 


t-4t 


and 


n  —  n  —  n    n-1    n 


for  n  =  2 N  where 


^n  =  ^K Vl'  V  I  ^\ Vl^  '  K-1'   \   '   V 


x'  = 


:^  =  {x  €  x"   I  y   a.(x.)    <  b.,  1  <  i  <  M} 
n        n   '  ^^^  ij   J     i' 


and 


X  =  {x  e  X   I  X  is  efficient  with  respect  to  X  }  . 
n        n  '  *^        n 

_  N      _     _  _ 

If  X  e  X   and  V   a^  .(x.)  =  B.  for  l<i<M,  then  x  is  an  optimal  solution 
N      j=i  iJ   J     i 

of  (1.1)  with  b  replaced  by  3.   This  follows  directly  from  the  definition 
of  dominance.   Thus  finding  all  efficient  solutions  of  (1.1)  for  right- 
hand-side  b  is  equivalent  to  finding  all  optimal  solutions  for  every  right- 
hand-side  b'  <  b. 

The  procedure  for  obtaining  X^  may  be  stated  quite  simply  as  follows: 

DP  Algorithm 
Step  1.   Set  n=l,  X  =  S^ 

Step  2.   Construct  X  by  eliminating  all  infeasible  elements  of  X  . 

e  f 

Step  3.   Construct  X  by  eliminating  all  dominated  elements  of  X  . 

0    e 
Step  4.   If  n  =  N,  stop.   Otherwise  set  n=n+l,  generate  X  =  X  ,  x  S  , 

'  ''        n    n-1    n 

and  go  to  Step  2. 

This  procedure  is  equivalent  to  an  imbedded  state  space  dynamic  programming 
algorithm   [ 2l]  and  is  similar  to  the  approaches  to  capital  budgeting  problems 
taken  In  [25]  and  [37].   The  feasibility  testing  (Step  2)  is  straightforward 
and  the  dominance  testing  (Step  3)  can  be  done  quite  efficiently  through  the 
use  of  (M+1)  threaded  lists,  as  described  in  [21]- 


-5- 

Upon   termination,   X^   is   at  hand  and   the   optimal  solution  of   (1.1)      for 

any   right-hand-side  b'   <  b     may  be  determined  by   inspection: 

N  N 

f^(b')    =  max   {   I        r   (x  )    |    x   e  X^  and     I        a      (x  )    <  b'     l<i<M}    . 
-1=1      J      J  i=l 

Notice   that   optimal  return   function   f   (b')    is   a  nondecreasing,    upper  semi- 
continuous   step   function  on  0<b'<b      [  12,    20,    21].      The  set   of  points   of   dis- 
continuity which,  completely  deteinuine  this   function  is 
N  N 


"^.,^13'V J.,  ^.j'V>  I  -  ^ 'S, '• 


The  "pure"  dynamic  programming  algorithm  just  presented  produces  all 
of  the  optimal  solutions  for  every  right-hand-side  b'  <  b.   Let  us  now 
restrict  our  attention  to  finding  an  optimal  solution  for  the  given  b-vector 
alone.   This  is  done  by  incorporating  elimination  by  bound  into  the  DP  frame- 
work.  In  Section  6  we  shall  indicate  how  a  parametric  analysis  on  the 
right-hand-side  of  (l-l)  niay  be  performed. 

Consider  any  x  =  (x, x  )  e  X  and  let 

1        n     n 

B  =  I       a^(x,) 
j=l     ^ 

where  a  (x.)  =  (a  .(x.),  ...,  a^.(x.))'.   We  may  interpret  g  as  the  resource 

consumption  vector  for  the  partial  solution  x.   The  residual  problem  at  stage 

n,    given  x,   is 

N 
^        (b-6)    =  max     I  r   (x  )  (2.2) 

j=n+l     J      J 

subject   to 

N 


U.  '^^'"j'  ''^-'^ 


l<i<M 


X.    e   S.  n+l<j<N 

J  J 
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Thus  f  ,  (b  -  3)  is  the  maximum  possible  return  from  the  remaining  stages, 
n+1 

given  that  resources  3  have  already  been  consumed.   For  each  0  <  n  <  N-1, 

'\, 
let  UB  ,  be  an  upper  bound  functional  for  f  ,,,  i.e. 
n+1  n+1' 

li   ^1  (b-B)  <  UB  _^T  (b-B)        for  all  0  <  6  <  b. 
n+1         n+1 

UB  ,1  may  be  taken  as  the  optimal  value  of  any  relaxation  of  the  residual 
n+1 

problem  (2.2).   We  assume  that  an  algorithm  is  available  for  solving  the 

chosen  relaxation.   Different  relaxations  may  be  used  at  different  stages 

and  UB    may  be  taken  as  the  smallest  value  obtained  by  solving  several 
alternative  relaxations.   (Let  UB^^   =   0). 

Any  known  feasible  solution  of  (1.1)  provides  a  lower  bound  on  f  (b) . 
The  best  of  the  known  solutions  will  be  called  the  incumbent  and  its  value 
denoted  LB,  so  that  LB  <  f^(b).   At  worst,  x  =  0  is  feasible  with  value  LB  =  0. 
These  upper  and  lower  bounds  can  be  used  to  eliminate  efficient  partial 
solutions  which  cannot  lead  to  a,   solution  that  is  better  than  the  incumbent. 
Tkat  Is,  If  X  e  X^  and 


4 

I        r  (x  )  +  UB    (b-  I        ahx.))    <   LB  (2.3) 

j=l  ^   ^  j=l     ^ 

then  no  completion  of  x  can  be  better  than  the  incumbent.   In  this  event 

we  say  that  x  has  been  eliminated  by  bound.   The  survivors  at  stage  n  will 

be  denoted  X  ,  where 
n' 

n  ^ 

X^  =  {x  e  X^  I   y    T(x.)+\JB^(h-l        aJ(x.))  >  LB}, 
n        n  I   ^^^  j   j'     n+1      ^^j^     J 

The  lower  bound  may  be  improved  during  the  course  of  the  algorithm  by 
finding  additional  feasible  solutions.   Assume  that  a  heuristic  is  available 
for  finding  good  feasible  solutions  and  let  H  ^  (b  -  B)  denote  the  objective 
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f unction  value  obtained  when  the  heuristic  is  applied  to  the  residual  problem 

(2.2).   (Let  H^  ,  =0).   If  (x'  ,  ,  ...,  x^)  is  the  completion  found  by  the 

s 
heuristic  for  (x,  ,  ....  x  )  e  X  ,  then  (x,  x')  is  feasible  for  (1.1)  and 
1'     '  n     n' 

becomes  the  new  incumbent  if 

n  n 

I        r.(x  )  +  H  ,(b  -  I        aJ(x  )  )  >  LB, 

j=l  J   ^     ""^^     j=l     J 

i.e.  if 

n  N 

I        r.(x,)  +  I  r  (x:)  >  LB. 

j=l  J  J    j=n+l  -■  ^ 

As  with   the  upper  bounds,    different  heuristics  may  be  used   at   different 
stages   and  H     ^    may  be   taken   as   the  largest  value  obtained  by  several 
competing  heuristics. 

At   the  end  of  stage  n  we  know  that   f>j(b)    falls  between  LB   and  the   global 
upper  bound 

n  n 

UB  =  max      {7        r^(x.)   +  UB  ^, (b  -   T        a^(x.))    I    x   e   X^    }. 

J    1      J      J  n+1  ^   T  J  n 

j=l     J      J  j=l  J 

If  the  gap  (UB-LB)  is  sufficiently  small,  then  we  may  choose  to  accept  the 
incumbent  as  being  sufficiently  close  to  optimality  in  value  and  terminate  the 
algorithm  rather  than  continuing  to  stage  N. 

To  incorporate  elimination  by  bound  into  the  dynamic  programming 

f      e 
procedure  we  must  redefine  X  and  X  as  subsets  of  the  feasible  and  efficient 

n      n    

solutions,  respectively,  and  redefine  X  as 

X°  -  X%  X  s 
n    n-1    n 

for  n=2 N.   Only  the  survivors  at  stage  (n-1)  are  used  to  generate 

potential  solutions  at  stage  n.   The  hybrid  algorithm  may  then  be  stated  as 
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follows.   In  the  terminology  of  [9],  partial  solutions  are  fathomed  if  they 
are  infeasible  (Step  2),  dominated  (Step  3),  or  eliminated  by  bound  (Step  5) 


Hybrid  Algorithm 

Step  1.   Set  n  =  1,  X^  =  S  ,  LB  =  H  (b) ,  UB  =  UB  (b);  choose  e  e  [0,1) 

and  L  >  1.   Stop  if  LB  =  UB. 

Step  2.   Construct  X  by  eliminating  all  infeasible  elements  of  X  . 
n  n 

e  f 

Step  3.   Construct  X  by  eliminating  all  dominated  elements  of  X  . 
n  °  n 

Step  4.   If  Ix^l  <  L,  set  X^  =  X^  and  go  to  Step  9. 
n '  n    n 

n  n 

Step  5.   Construct  X^  =  {x  e  X®  |  )]   r.(x.)  +  UB    (b  -  I       a-^(x.))  >  LB}. 

'^   j=l  J   3      ^  j=l     3 

n  n 

Step  6.   UB'  =  max  {T   r .  (x.)  +  UB  ^,  (b  -  J   a^(x.))  I  x  e  X^},  and 

•  1  2      2  n+1     'r  T     1  n 

3=1  J   J  j=3^     J 

UB  =  min  {UB,  UB'}. 

n  n 

Step  7.   LB  =  max  {^   r.(x.)  +  H   (b  -  I       a^(x.))  |  x  e  X^}, 
j=l  ^   J     "       j=l     J  ^ 

LB  =  max  {LB,  LB'},  change  the  incumbent  if  necessary. 

Step  8.   If  (UB  -  LB)/UB  <  e,  stop.   The  incumbent  is  sufficiently  close  to 

an  optimal  solution  in  value. 

g 

Step  9.   If  n  =  N,  stop:  either  X^  contains  an  optimal  solution  of  the  incumbent 

0    s 
is  optimal.   Otherwise,  set  n  =  n+1,  generate  X  =  X  .  x  S  ,  and  go 

n    n-1    n 

to  Step  2. 


The  parameter  £  determines  the  approximation  to  optimality,  with  e=0 

corresponding  to  exact  optimality.   For  e>0  we  have  LB>(l-e)UB>(l-e)f  (b)  when 

N 
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termlnatlon  occurs  at  Step  8.   Note  that  an  early  stop  at  Step  8  may  occur 
even  for  the  e=0  case  if  IJB=LB.   To  find  all  of  the  alternative  optimal 
solutions  for  right-hand-side  b,  use  "  >LB  "  rather  than  "  >LB  "  at  Step  5 
and  choose  e=0. 

If  L=l,  then  upper  and  lower  bounds  will  be  computed  at  every  stage. 
Our  empirical  evidence  indicates  that  the  total  amount  of  computation  required 
may  be  substantially  reduced  if  these  bounds  are  determined  only  inter- 
mittently.  This  could  be  done  at  every  k   stage  or,  as  shown  here,  whenever 
the  number  of  efficient  partial  solutions  exceeds  a  specified  limit  L.   As 
long  as  this  niimber  remains  less  than  L,  we  just  use  the  trivial  upper 

bound  (UB  .^  E  +  =»)  and  the  trivial  heuristic  (H  .^  =0)  which  yield 
n+1  n+i 

X"  =  X^  . 
n    n 

It  appears  from  the  statement  of  Step  5  that  UB  , ,  (b  -  2,   a  (x.))  must 

e  "^       j=l     J 

be  computed  independently  for  each  x  e  X  .   It  will  be  shown  in  the  next 

section  that  this  is  not  the  case.   In  fact  the  attractiveness  of  this  hybrid 

approach  stems  largely  from  the  ease  with  which  information  about  the 

UB  .,(•)  function  can  be  shared  among  the  elements  of  X  . 
n+1  n 

The  progress  of  the  algorithm  may  be  represented  by  a  tree  of  partial 

solutions.   Figure  1  shows  the  first  three  stages  of  a  problem  with  K  =2 

and  K  =K  =1.   None  of  the  nodes  at  stage  1  are  eliminated,  so  X  =  {x  ,  ...,  x  }, 

If  we  suppose  that  x  is  infeasible,  that  x  is  dominated  by  x  ,  and  that  x 

is  eliminated  by  bound,  then  X„  =  {x  ,x  ,x  ,x  ,x  },  X„  =  {x  ,x  ,x  ,x  }, 

s     2   3  4 
and  the  nodes  at  stage  3  are  the  descendants  of  X  =  {x  ,x  ,x  }.  All  of 

the  nodes  that  must  be  considered  at  stage  n  are  generated  before  any  node 

at  stage  (n+1).   There  is  no  backtracking.   This  is  in  marked  contrast  to 

conventional  branch-and-bound  methods  where  one  typically  finds  active  nodes 

at  many  different  levels  in  the  tree. 


=  0 


=  0. 


Figure  1.   Tree  of  partial  solutions,  first  three  stages, 
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3.   Relaxations  for  Upper  Bounds 

Our  development  of  the  hybrid  algorithm  has  assumed  the  availability 
of  algorithms  for  solving  relaxations  of  (2.2)  and  of  heuristics  for  finding 
feasible  solutions  of  (2.2).   In  this  section  and  the  next  we  present  some 
of  the  relaxations  and  heuristics  that  are  appropriate  in  this  context 
and  that  we  have  tested  computationally. 

Solving  any  relaxed  version  of  the  residual  problem  (2.2)  yields  a  valid 

upper  boimd.   The  simplest  relaxation  is  to  drop  all  of  the  constraints. 

This  gives 

N 
UB„.i(b  -  B)  =  I     r  (K  )  (3.1) 

^■^^         j=n+l  J   J 

which  is  independent  of  3.   A  less  drastic  relaxation  is  to  keep  just  one 
constraint,  say  constraint  i.   The  "best  remaining  ratio"  for  constraint  i  at 
stage  n  is 

BRR.   ^,  =  max     {max  {r . (k) /a^  . (k)  I  k=l,  ....  K.}} 
^'^■'^   n+l<j^N        J     iJ     '     '     *   J 

where  the  ratio  is  taken  as  +  "  if  a   (k)  =  0.  An  upper  bound  based  on 
constraint  i  is: 

UBi^l(b  -  6)  =  (b^  -  6^)  *  BRR^^^^^ 

and  if  this  is  computed  for  each  i=l,  ...»  M  then  we  also  have 

UB  ,,  (b  -  B)  =  min   UB^_^,  (b  -  B).  (3.2) 

^"^^         l<i<M   """^ 


-11- 


Note  that  the  best  remaining  ratios  can  be  tabulated  in  advance  for  l<i<M  and 
0^n<N-l  so  that  the  maximizations  are  done  only  once. 

These  upper  bounds  are  useful  and  very  simple  to  compute,  but  they  are 

a. 
quite  weak.   They  generally  overestimate  f   (b  -  3)  by  a  wide  margin.   To 

obtain  stronger  bounds  we  must  resort  to  linear  programming.   Let  us 

consider  first  the  case  where  (1.1)  is  a  linear  integer  program,  i.e. 

r.(x.)  =  r.x.  and  a. .(x.)  =  a. .x.  for  all  i,i.   The  continuous  relaxation  of 

(2.2)  is  then  a  linear  program  whose  value  may  be  taken  as  UB   (b  -  3) • 

N 
UB  ^, (b  -  B)  =  max  T     r.x.  (3.3) 

^^^  j=n+l  J  J 

subject  to 


N 

y     a. .X.  <  b.  -  B^    l<i<M 


OSx.<K,  n+l<i<N 

J   J 


This  linear  program  has  a  finite  optimal  solution  for  every  0<6<b  since 
x=0  is  always  feasible  and  all  of  the  variables  have  upper  bounds.   By 
linear  programming  duality,  then,  we  may  write 


M  N 

UB  .^(b  -  fl)  =  min  y   u.(b.  -  6.)  +  y     v.K.  (3.4) 

x=l  J=n+1  -"  -^ 


subiect  to 
M 


y   u, a^.  +  v^  >  r.        n+l<j<N 


u  >  0         l<i<M 
V.  >  0       n+l<iSN 
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We  propose  to  use  linear  programming  in  a  way  that  is  quite  different 

from  the  usual  practice  in  branch-and-bound  methods  [  8 ] •   Our  approach  is 

based  on  the  fact  that  the  residual  problems  corresponding  to  the  partial 

solutions  in  X  are  identical,  except  in  their  right-hand-sides.   This  makes 
n 

it  possible  to  obtain  bounds  for  all  of  these  problems  simultaneously,  as  will 
now  be  demonstrated. 

Let  X  =  {x  ,  X  X  }  and  let  the  corresponding  resource  consumption 

12        0 
vectors  be  g  ,  g  ,  . . . ,  3  where 


6?  =  I        a  x^  for  i=l,  . . . ,  M. 

j=l   J  -■ 

The  feasible  region  of  the  dual  problem  (3.4)  is  a  non-empty,  unbounded 

polyhedron  which  will  be  denoted  D   .   Let  { (u  ,  v  )  |  t  e  '^^+1'^  ^^ 

the  set  of  extreme  points  of  D   .   Since  32b,  (3.4)  achieves  its  minimum  at 

an  extreme  point  of  D  ^    and 


3^^^(b  -  6)  =  min  I        .[   (b^  -  3.)  -H  I     vJk 
i=l  j=n+l    -^ 


n+1 
for  0<3<b.   It  follows  that  for  q=l Q  we  have 

M  N 

m    .Ah  -   &^)    ^   I        u5  (b,  -  6?)  +  y     viK,  (3.6) 


B„+i (b  -  B^)  2  I   uj  (b  -  bJ)  +  ^     v^K 
n+1  i=l  ^   ^    ^    j=n+l  J  J 


for  all  t€T  , , . This  means  that  any  dual  extreme  point  can  be  used  to  perform  a 
n+1  — i- 

bounding  test  on  every  element  of  X  .   Combining  (2.3)  and  (3.6)  we  see  that 
x*^  is  eliminated  by  bound  if 

j=l  -^  -^   i=l  J=n+1  -'  ■' 

for  some  teT 

n+1 
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To  exploit  this  opportunity  for  sharing  dual  solutions  among  the  elements 
of  X  ,  we  propose  a  parametric  tour  of  6-space  which  visits  g  ,  6  ,  ... ,  B  . 

Suppose  that  (3.4)  has  been  solved  for  3=3   and  that  we  are  in  the  process  of 

2 
obtaining  an  optimal  solution  for  3=3  by  parametric  linear  programming: 

1       2    1 
3=3  +  X  (3  -  3  )  for  0<X<1.   At  each  iteration  (dual  simplex  pivot)  we  move  to 

2 
a  new  dual  extreme  point  and  have  a  new  opportunity  to  eliminate  not  only  x 

but  also  X  ,  ...,  X  .   If  X  is  eliminated,  then  3  may  be  dropped  from  the 

itinerary  of  the  tour.   The  details  of  such  a  strategy  are  spelled  out  in 

the  following  "resource-space  tour"  procedure,  which  may  be  used  at  Step  5  of  the 

hybrid  algorithm.   Upon  completion  of  the  tour,  X^  =  {x^  e  X   |  s(q)  =  1}  . 


Resource-Snace, Tour 


Step  1.   Set  s(q)  =  1  for  q  =  1 Q.   Solve  (3.4)  for  3  =  3  .   If 

n      ,  , 

y   r.x;:  +  UB  ^,  (b  -  3  )  ^  LB 

eliminate  x  bv  settine  s(l)  =  0.   Set  p  =  1. 
Step  2.   Set  3*  =  B^.   3*  is  the  starting  point  for  the  next  parametric  segment. 

Step  3.   If  p  =  Q  or  s(q)  =  0  for  all  q  >  p,  stop.   Otherwise  set 

c  =  min  {q>p  |  s(q)  =  1}.   3  is  the  destination  of  the  next 

parametric  segment. 
Step  4.   Use  parametric  programming  on  (3.4)  with  3=3*  +  A(3  -  3*)  to  drive 

X   from  0  to  1.   At  each  basis  change,  A  =  A,  use  the  dual  solution 

(u,  v)  to  execute  Steps  5  and  6. 
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Step  5.   For  q  =  c,  c  +  1,  ...,  Q:  if  s(q)  =  1  and 

n  M   _  N     _ 

I        r  x^  +  I        u   (b  -  bJ)  +  I     V  K  <  LB, 

j=l  ^  ^   i=l  ^   ^    ^    j=n+l  ^  ^ 

eliminate   x  by  setting  s(q)  =  0. 

Step  6.   If  T  =  1,  set  p  =  c  and  go  to  Step  2.   If  X<1  but  s(c)  =  0,  set 

3*  =  B*  +  X  (b'^  -  B*)  ,  set  p  =  c,  and  go  to  Step  3.   Otherwise 

continue  with  Step  4. 

2 
Figure  2  illustrates  the  possible  outcome  of  such  a  parametric  tour  for  SeR  • 

2 
The  x's  mark  the  successive  basis  changes.   The  path  shown  would  result  if  x 

were  eliminated  by  the  dual  solution  obtained  at  A  and  x  were  eliminated 

by  the  dual  solution  obtained  at  B.   We  shall  use  the  term  direct  hit  to 

2         2 
describe  the  elimination  of  x  ,  since  3  was  the  destination  of  the  current 

parametric  segment,  and  indirect  hit  to  describe  the  elimination  of  x  . 

The  computational  advantage  achieved  by  the  resource-space  tour  is  primarily 

because  of  the  frequent  occurence  of  indirect  hits.   The  partial  solutions 

e 
in  X  share  dual  solutions  and  therefore  share  the  computational  burden  of 

the  simplex  pivots. 

e 

In  the  results  to  be  reported  here,  the  elements  of  X  were  always 

ordered  according  to  their  objective  function  value,  i.e. 

I        r  x^  <  I        r  xf^   for  q  =  1 Q  -  1. 

j=l  -■  -■   j=l  -■  -■ 

We  are  not  currently  aware  of  any  more  compelling  criterion.   Notice 

that  an  optimal  LP  solution  is  obtained  for  each  survivor  in  X   .   If  xeX 

n         n 

and  X*  is  the  optimal  LP  solution  for  the  corresponding  residual  problem,  then 

3 

X  may  be  dropped  from  X  if  x*  is  all  integer.   The  complete  solution  (x,  x*) 
n 

becomes  the  new  incumbent. 
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2 
Figure  2,   A  typical  resource-space  tour  in  R  . 
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When  problem  (1.1)  Is  nonlinear,  we  may  still  use  linear  programming 
to  compute  strong  upper  bounds.   For  each  variable  x.,  if  K,>1  and  some  of 
the  functions  r.(*).  a,  .(•)>  •••»  &.,.(•)  are  nonlinear,  then  we  call  x.  a 
nonlinear  variable  and  "expand"  it  into  the  binary  variables 

\  if  X.  =  k 
^jk    \  ^  (3.8) 

0  otherwise 

for  k=0,  1,  ...,  K..   The  following  multiple  choice  constraint  on  the  y 

will  insure  that  x.  assumes  one  of  its  permissable  integer  values: 

K. 

k=0  ^^ 


If  all  of  the  variables  are  nonlinear,  then  (1.1)  is  equivalent  to  the 
following  zero/one  integer  linear  program  with  multiple  choice  constraints: 

subject  to 
N    K. 

j=l  k=0   ^^^  ^^  ^ 

K. 
k=0   ^^ 


y.,  e  {0,1}         l^j^N,  0<k<K. 

where  r  =r.(k)  and  a   =a   (k) .   (In  general  only  the  nonlinear  variables 
jk  J         Ijk  ij 

would  have  to  be  expanded.) 

When  (3.10)  is  relaxed  to  a  linear  program,  the  simple  upper  bounds 
(y  <1)  may  be  dropped  since  they  are  implied  by  the  multiple  choice  constraints, 
This  is  important  since  it  means  that  we  do  not  have  explicit  dual  variables 
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for  them.   The  multiple  choice  constraints  may  be  handled  implicitly  as 
generalized  upper  bounds  (GUB's).   Thus  in  the  nonlinear  case  we  have: 

M  N 


UB^_^^(b-6)  =  min  I       u^(b^-6.)  +  1     v 
i=l  j=n-l-l  -^ 


subject 

to 

M 

I 
i=l 

u  a       +v.sr ., 

u  >0 

v,>0 

2 
l<i<M 
n+l<j<N. 


The  (v  ,  ..,,  V  )  are  now  the  dual  variables  for  the  GUB  constraints,  and 
the  resource-space  tour  may  be  performed  exactly  as  described  above. 

The  use  of  transformation  (3.8)  rather  than  the  traditional  binary 
expansion  leads  to  the  introduction  of  a  GUB  constraint  (3.9)  rather  than 
a  general  linear  constraint  to  impose  the  upper  bound  K..   The  practical 
importance  of  this  was  recently  pointed  out  by  Glover  [11 J . 
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4.  Heuristics  for  Lower  Bounds 

There  are  several  effective  heuristics  which  may  be  applied  to  linear 
problems.   These  include  Senju  and  Toyoda  [35],  Toyoda  [36],  and  Petersen  [28]. 
The  latter  two  have  been  incorporated  into  the  computer  code  for  the  hybrid 
algorithm  and  their  performance  will  be  discussed  in  Section  5.   We  have  also 
obtained  good  integer  solutions  by  rounding  down  LP  solutions.   These  integer 
solutions  may  then  be  improved  by  re-allocating  the  resources  freed  by 
rounding  down.   That  is,  we  may  increase  by  one  any  variable  that  is  currently 
below  its  upper  bound  and  that  consumes  no  more  than  the  leftover  resources. 
This  may  be  repeated  until  there  is  no  such  variable  remaining. 

In  the  nonlinear  case,  heuristics  may  be  applied  directly  to  (1.1) 
or  to  its  linear  representation  (3.10).  If  y*  is  the  optimal  LP  solution 
of  (3.10)  and 

K-    * 

X*  =  l^      ky  l<j<N, 

J     k=0   ^^ 
then  rounding  down  x*  may  not  result  in  a  feasible  solution  of  (1.1). 
(The  same  is  true  for  any  residual  problem.)   In  this  event,  the 
components  of  x*  may  be  reduced  one  at  a  time  until  a  feasible  solution  is 
obtained.   At  worst  this  will  be  x=0.   Then  a  re-allocation  procedure  similar 
to  the  one  described  above  may  be  applied. 
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For  linear  or  nonlinear  problems  the  following  "myopic"  heuristic 
is  useful.   Consider  the  variables  x  ^,  ...,  x^  in  order.   For  each  one 
determine  the  largest  feasible  value  it  can  assume,  given  the  values  chosen 
for  the  preceeding  variables.   That  is 

X  ,1  =  max  {x  .^eS  ,T  I  a    (x  ,, )  <  b  -  3^} 
n+1        n+1  n+1  '        n+1 


and 

p=n+l 


X.  =  max  {X.6S.  |  a^Cx.)  <  b  -  B^^  -  J     a^Cx^)} 


for  j  =  n+2 N.   Then  x  is  feasible  for  (2.2)  and 


N 


H^,  (b-  B^)  =  I  r  A).  (4.1) 

"+1  j=n+l  J   ^ 


Various  "greedy"  heuristics  could  also  be  used,  see  for  example  Magazine, 
Nemhauser  and  Trotter  {17]. 
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5.   Computational  Results 

The  hybrid  algorithm  has  been  tested  on  a  set  of  capital  budgeting 
problems  taken  from  the  literature.   Problems  1  and  2  are  among  those  solved 
in  [28].   Problems  3  and  4  are  problems  7  and  5,  respectively,  of  Petersen 
[27].   Problems  5  and  6  are  constructed  from^ parts  of  Problems  1,  2,  3, 
and  4.   (Problem  5  is  a  subset  of  Problem  6).   Problems  9  and  10  are  the 
30x60  problem  of  Senju  and  Toyoda  [35]  with  their  right-hand-sides  A  and  B, 
respectively.   (A  is  60%  of  B.)   Problems  7  and  8  have  the  first  30  columns 
of  the  Senju  and  Toyoda  problem  and  half  of  right-hand-sides  A  and  B,  respectively. 

These  problems  are  all  of  the  zero/one  knapsack  type  —  i.e.  they 
satisfy  the  non-negativity  assumption.   The  coefficient  matrices  are  all  at 
least  90%  dense  in  non-zero  elements.   All  of  the  problems  were  solved  to 
exact  optimality  (e=0) .   Prior  to  solution  the  columns  were  sorted  into  non- 
increasing  order  of  their  objective  values  and  renumbered.   Thus: 

r^.r^^  ...  .r^. 

Four  heuristics  were  employed:   Petersen  [28j»  Toyoda  l36]>  Rounding 
and  Myopic  —  the  latter  two  as  described  in  Section  4.   The  Petersen  heuristic 
gave  the  best  results,  but  was  also  the  most  time  consuming.   (We  used  only 
the  First  Search  and  Fitback  procedures.)   For  this  reason  Petersen  was  used 
only  once  on  each  problem,  at  the  top  of  the  search  tree  (stage  0) .   Toyoda, 
Rounding  and  Myopic  were  applied  to  every  survivor  of  the  resource  space  tour. 

The  resource  space  tour  was  made  only  when  the  number  of  partial  solutions 
exceeded  the  threshold  L.   (All  of  the  LP  computations  were  performed  by  sub- 
routines of  the  SEXOP  system  [18].)   The  r(K)-bound,  (3.1),  and  the  BRR-bound, 
(3.2),  were  used  at  every  stage  since  they  could  be  applied  so  cheaply. 

Tables  lA  and  IB  summarize  our  experience  with  these  zero /one  problems. 
The  "Values"  section  of  the  table  records  the  continuous  and  integer  optimal 
values  as  well  as  the  initial  lower  bounds  obtained  by  the  Petersen  and 
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Rounding  heuristics.   The  "Improvements"  section  gives  the  number  of  improved 
feasible  solutions  discovered  by  each  heuristic.   The  "Eliminations"  section 
records  the  number  of  nodes  (partial  solutions)  eliminated  by  each  of  the 
several  techniques .   Those  eliminated  by  the  resource-space  tour  are  divided 
into  direct  hits  and  indirect  hits,  as  described  in  Section  3.   The  "LP  &  H" 
row  gives  the  number  of  stages  at  which  LP  and  the  heuristics  were  invoked, 
i.e.  the  number  of  times  there  were  more  than  L  efficient  partial  solutions. 
The  threshold  L  was  set  at  100  for  Problems  1-9  and  at  200  for  Problem  10. 
"Projects  selected"  is  the  number  of  ones  in  the  integer  optimal  solution. 
The  computation  time  is  in  CPU  seconds  for  an  IBM  370/168. 

The  Petersen  heuristic  was  quite  effective  on  these  problems,  usually 
within  1%  of  the  optimum.   To  see  how  the  algorithm  would  fare  without  such 
a  good  initial  lower  bound,  we  ran  problems  5,  6,  and  7  with  and  without  the 
Petersen  heuristic.   In  the  latter  case  the  Rounding  value  was  used  as  the 
initial  lower  bound.   The  computation  times  were  greater  without  Petersen, 
but  not  dramatically  so.   The  other  three  heuristics  were  able  to  bring  the 
lower  bound  up  to  or  above  the  Petersen  value  very  quickly.   The  results 
illustrate  the  value  of  having  a  diverse  collection  of  heuristics. 

One  of  our  chief  surprises  in  experimenting  with  the  hybrid  algorithm 
was  that  the  LP  bounds  and  heuristics  would  be  invoked  so  few  times.   Table  2 
summarizes  a  series  of  runs  on  Problem  2  which  compare  different  values  of  the 
threshold  L.   It  is  apparent  that  when  LP  is  used  only  intermittently  the 
weaker  bounds,  and  dominance,  play  a  much  larger  role.   Not  using  LP  for 
several  stages  causes  us  to  accumulate  a  great  many  unattractive  partial 
solutions  that  would  have  been  fathomed  by  LP.   Some  of  them  are  so  unattractive 
that  they  can  be  fathomed  by  the  weak  r(K)  and  BRR-bounds.   The  ones  that 
survive  are  eliminated  very  efficiently  by  indirect  hits  when  LP  is  finally 
called  in  for  a  "clean  up".   This  was  our  other  chief  surprise  -  the  frequency 
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of  indirect  hits.   Table  2  shows  clearly  that  the  relative  number  of  indirect 
hits  increases  with  the  threshold  L  as  more  and  more  unattractive  nodes  are 
allowed  to  accumulate.   It  was  quite  common  for  the  number  of  partial  solutions 
to  drop  from  over  100  to  less  than  10  when  LP  was  applied  -  with  most  of  the 
eliminations  being  by  indirect  hits. 

We  have  also  investigated  the  effect  of  allowing  the  variables  to 
assume  values  in  {0,  1,  2}  or  {0,  1,  2,  3}.   The  increase  in  computing  time 
can,  of  course,  be  very  great.   For  a  fixed  level  of  resources,  however, 
there  may  be  a  great  deal  of  elimination  by  infeasibility  when  K  is  increased 
from  1  to  2  or  3.   This  is  illustrated  in  Table  3.   The  first  three  columns 
represent  Problem  1  solved  for  K=l,  2,  3  respectively.   The  b-vector  was  the 
same  in  each  case.   The  fourth  column  is  the  K=3  case  repeated  with  the 
original  b-vector  doubled.   To  promote  comparability  the  Petersen  and  Toyoda 
zero/one  heuristics  were  not  used  in  the  K=l  case.   A  threshold  of  L=50  was 
used  in  all  4  runs. 

Table  4  reports  the  results  of  some  experiments  on  nonlinear  variations 
of  Problem  1.   In  each  case  the  variables  were  allowed  to  assume  values  in 
{0,  1,  2}.  The  convex  objective  for  run  2  was  r.(2)=4r  (1)  for  all  j;   the 
convex  constraints  for  run  3  were  a. . (2)=4a, , (1)  for  all  i,  j;   and  run  4  had 
r.(2)=4r.(l)+10  and  a   (2)=4a   (1)+10  for  all  i,  j.   The  threshold  was  L=50 
for  all  runs  and  only  the  Rounding  and  Myopic  heuristics  were  used.   The 
computation  time  was  increased  by  a  factor  of  from  2  to  5  over  the  linear 
case  (run  1).   This  is  due  to  a  weakening  of  both  the  heuristics  and  the  LP 
bounds. 
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TABLE  2.  THE  EFFECT  OF  THE  THRESHOLD  L  ON  PROBLEM  2  (5x45) 


L 

1 

25 

50 

100 

LP  &  H  times 

44 

10 

6 

4 

Eliminations: 

Feasibility 

13 

20 

54 

76 

Dominance 

2 

36 

70 

130 

r(K).    (3.1) 

82 

119 

190 

353 

BRR,   (3.2) 

3 

11 

18 

50 

Direct  hits 

60 

29 

26 

24 

Indirect  hits 

114 

217 

316 

378 

Total   LP  pivots 

419 

243 

210 

184 

Time  (sec.) 

10.987 

5.131 

3.585 

3.360 

TABLE  3.  THE  EFFECT  OF  THE  NUMBER  OF  CHOICES  AT  EACH 
STAGE,  K,  ON  PROBLEM  1   (5x30). 


K 

1 

2 

3 

3 

Values: 

LP  optimum 

7700.53 

8725.44 

9131.90 

16,754.38 

Integer  optimum 

7515.00 

8451.00 

8920.00 

16,570.00 

Rounding 

7265.00 

8135.00 

8824.00 

16,168.00 

Improvements: 

Rounding 

1 

2 

2 

6 

Myopic 

3 

3 

1 

2 

El  iminations: 

Feasibility 

30 

218 

400 

247 

Dominance 

19 

47 

9 

12 

r(K),    (3.1) 

101 

74 

85 

93 

BRR,   (3.2) 

9 

22 

85 

52 

Direct  hits 

27 

28 

15 

68 

Indirect  hits 

101 

297 

343 

735 

LP  &  H  times 

3 

5 

4 

13 

Initial   LP  pivots 

40 

40 

46 

36 

Total   LP  pivots 

232 

323 

154 

1107 

Time  (sec. ) 

2.224 

3.493 

1.786 

14.308 

TABLE  4.  NONLINEAR  INTEGER  PROGRAMS,  VARIATIONS  ON  PROBLEM  1  (5x30), 


K 

2 

2 

2 

2 

Objective 

linear 

convex 

linear 

convex 

Constraints 

linear 

1 i  near 

convex 

convex 

Values: 

LP  optimum 

8725.44 

17,450.88 

7700.53 

9011.91 

Integer  optimum 

8451.00 

16,385.00 

7515.00 

8419.00 

Rounding 

8135.00 

13,305.00 

7211.00 

6626.00 

Improvements: 

Rounding 

2 

2 

0 

0 

Myopic 

3 

5 

3 

9 

El iminations: 

Feasibility 

218 

274 

151 

219 

Dominance 

47 

109 

45 

10 

r(K),    (3.1) 

74 

104 

66 

57 

BRR,   (3.2) 

22 

48 

6 

36 

Direct  hits 

28 

120 

109 

27 

Indirect  hits 

297 

170 

120 

170 

LP  &  H  times 

5 

5 

4 

3 

Initial   LP  pivots 

40 

51 

72 

49 

Total   LP  pivots 

323 

1022 

822 

432 

Time  (sec.) 

3.493 

15.844 

11.565 

6.408 
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6.   Parametric  Integer  Programming 

Parametric  integer  programming  has  only  recently  emerged  as  a  topic 
of  research.   The  pioneering  papers  include  Noltemeier  [26],  Roodman  [32  .33  ]  > 
Piper  and  Zoltners  [29],  and  Bowman  [3  ].   Nauss  [23]  has  reviewed  this 
earlier  work  and  contributed  many  new  results  for  parameterizations  of 
the  objective  function.   The  results  of  this  section  are  a  contribution 
to  the  right-hand-side  case.  See  also  [2,  10,  30]. 

The  hybrid  algorithm  of  Section  2  can  be  extended  to  solve  a  family 
of  parametric  integer  programs  whose  right-hand-sides  lie  along  a  given 
line  segment.   This  family  may  be  written  as: 

N 

I 
j  = 

subject  to 


g(e)  =  f  (b+6d)  =  max  I        r  (x  )  '    (6.1) 

1=1  -^   -^ 


N 


I       a,j(.j).b^^6d^ 


l<i<M 


x^eS.  l<j<N 

i      2 

where  0<e^l  and  d  =  (d  ,  ....  d^)  is  a  direction  vector.   To  "solve"  (6.1) 
is  to  obtain  an  optimal  solution  for  each  program  in  the  family,  i.e.  for 
each  e  e  [0,1].   In  solving  (1.1)  the  hybrid  algorithm  eliminates  partial 
solutions  for  reasons  of  feasibility,  dominance,  and  bound.   In  order  to 
solve  (6.1),  the  feasibility  and  bounding  tests  must  be  modified. 

A  partial  solution  x*^  should  be  eliminated  as  infeasible  only  if  it 
is  infeasible  for  all  0<e^l.   If  d2:0,  then  we  simply  use  (b+d)  rather  than 
b  in  the  feasiblity  test.   In  general  (d  ?=  0)  ,  let  9^  and   6^  denote  the 
smallest  and  largest  values  of  e€(-  ",  +  •") ,  respectively,  for  which  the 
residual  problem  for  x  is  feasible.   Then: 
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e^  =  max  {(3^  -  b^)/d^  i  d^>0}  (6.2) 

e^  =  min  {(bJ  -  b^)/d^  1  d^<0}  (6.3) 


and  X  can  be  eliminated  if  3!r>b   for  some  constraint  i  with  d.=0,  or 
e^>e^,  or  [e^,  e^]  n  [0,1]  =  ((>.   The  partial  solution  x'^  is  kept  if  [6^,  6^] 
has  a  non-empty  intersection  with  [0,1].   If  6^<b  and  d>0,  then  6^^0, 
e''  -  +  00 ,  and  the  intersection  is  exactly  [0,1]. 

A  partial  solution  x  should  be  eliminated  by  bound  only  if  it 
can  be  shown  that  none  of  its  descendants  is  optimal  for  any  ee[0,l].   In 
order  to  make  this  kind  of  test  the  upper  and  lower  bounds  used  by  the 
hybrid  algorithm  must  be  viewed  as  functions  of  6.   The  relationship 
between  the  incumbent  value  and  the  optimal  value,  LB<f  (b) ,  must  be 
extended  over  the  interval  of  parameterization:   LB(0)Sg(9).   It  was 
pointed  out  in  Section  2  that  f  (b")  is  a  nondecreasing  step  function  on 
b'eR  .   The  g(e)  function,  since  it  is  a  "slice"  of  f  (b'),  is  also  a  step 
function  and  is  nondecreasing  if  d^O.   Each  of  the  optimal  solutions  of 
(6.1)  determines  one  of  the  steps  of  g(6).   The  heuristic  at  Step  7  of 
the  hybrid  algorithm  can  be  used  to  construct  the  necessary  lower  approxim- 
ation of  g(6) .   Instead  of  computing  only  H   (b-g  )  we  shall  compute 

H  , ^ (b+ed-g")  for  several  values  of  9.   Then  LB(0)  is  defined  as  the  maximum 

n+i 

return  achieved  by  any  known  solution  whose  resource  consumption  vector 
does  not  exceed  (b+Od) .   LB(9)  is  a  step  function  (nondecreasing  if  dsO) 
and  LB(9)<g(9)  on  0^9:21.   The  solutions  which  determine  LB(9)  are  called  the 
incumbents.   Each  is  incumbent  for  a  particular  interval  of  0.   (Note  that 
all  of  these  step  fimctions  are  upper  semi-continuovis)  . 
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Let  us  specify  that  the  upper  bounds  at  Step  5  of  the  hybrid 
algorithm  are  to  be  computed  by  linear  programming.   Taking  advantage 
of  LP  duality  once  again,  we  have  for  0<6<1: 


f  ^,  (b+ed-e'^)  S  UB  ^,  (b+ed-B^) 
n+l  n+l 


M  N 

^  I        u^(b  +ed  -6^)  +  I  v^K 

i=l  i=n+l  ^  ^ 


for  any  teT  ^  .   The  return  that  can  be  achieved  by  any  descendant  of  x 

is  therefore  bounded  above  by  the  linear  function  of  6: 

M 
h(e;q,t)  =  [  I     u  d  ]  e 
i=l 

j=l  ^   J    1=1^"-'-    j=n+l  J  J 

where  (u  ,  v  )  is  any  extreme  point  of  the  dual  feasible  region  D   .   The 

tightest  bounds  for  x  would  of  course  be  those  based  on  dual  solutions 

that  are  optimal  for  (b+ed-B*^)  ,  Osesl.   Notice  that  h(e;q,t)  is  nondecreasing 

in  0  if  d>0  since  u*'iO. 

The  new  bounding  test  is  of  the  form:   eliminate  x^eX^  if 

n 

UB^_^^(b+ed-B'^)  <:   LB(e)   for  O<0<1.  (6.4) 

A  sufficient  condition  for  eliminating  x  is  therefore 

h(e;q,t)  i   LB(e)    for  056^1.  (6.5) 

for  some  teT  -.   Thus  (6.5)  is  the  appropriate  generalization  of  the 

single  point  test  (3.7).   Figure  3  illustrates  a  successful  bounding  test 

in  a  case  where  d  >0.   If  (6.5)  is  satisfied  as  a  strict  inequality  for  all  6, 
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Figure  3.  A  successful  bounding  test. 
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then  X  does  not  have  any  optimal  descendants.   If  (6.5)  is  satisfied  but 
holds  with  equality  for  some  6,  then  one  of  the  incumbents  may  be  a  descendant 
of  X  and  may  be  optimal  for  6.   We  may  safely  eliminate  x  ,  however, 
since  it  cannot  have  any  descendant  that  is  better  than  an  incumbent. 
Notice  that,  unlike  ordinary  integer  programming  (d=0) ,  an  all-integer  LP 
solution  for  some  0e[O,l]  does  not  in  itself  justify  dropping  x  . 

The  test  (6.5)  may  be  strengthened  by  narrowing  the  interval  over 

Q   e 
which  the  inequality  must  hold.   If  x  eX  ,  then  we  know  that  the  residual 

n 

problem  for  x  is  infeasible  outside  of  the  interval  [6^,  69]  » 

where  9^  and  6^  are  given  by  (6.2)  and  (6.3), 

respectively.   Since  UB   (b+ed-B*^)  =  -  «=  for  9f[e?,  6^]  and  this  clearly 

satisfies  (6.4),  the  test  (6.5)  may  be  refined  to  :   eliminate  x  eX  if 

h(e;q,t)  ^  LB(6)    for  ee[e^,  9^]  n  [0,1]  (6.6) 

for  some  teT 

n+1 

The  execution  of  the  bounding  tests  at  Step  5  can  be  organized  in 
several  different  ways.   The  simplest  would  be  to  solve  each  linear  program 
independently  and  make  just  one  test  for  each  one,  the  test  for  x  being 
based  on  the  dual  solution  that  is  optimal  for  (b-g  )  ,  or  for  (b+OM-g  ) 
if  0<  B^   Si.   If  this  test  did  not  succeed  in  eliminating  x  ,  then  parametric 
linear  programming  could  be  xised  to  generate  some  or  all  of  the  dual  solutions 
that  are  optimal  for  (b+ed-g*^)  ,  Be [6^,  6^]  n  [0,1].   Let  these  be  (u  ,  v  )  , 

t=l,  ...,  P.   A  stronger  test  can  then  be  based  on  the  pointwise  minimum 

q   e 
of  the  several  linear  functions:   eliminate  x  eX  if 

n 

min        h(e;q,t)  ^  LB(9)     for  0e[e^,  6^]  n  [0,1].     (6.7) 
t=l  ..   P  1   ^ 
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This  pointwise  minimum  is  the  familiar  concave  piecewise  linear  function 
of  6  associated  with  parametric  linear  programming. 

As  in  Section  3,  the  idea  of  a  €our  through  resource-space  is 
much  more  attractive  than  treating  each  linear  program  independently. 
Since  the  h-f unctions  are  based  on  dual  feasible  solutions,  the  parametric 
tour  can  be  carried  out  exactly  as  in  Section  3.   Only  the  bounding  test 
(6.6)  is  different,   in  addition,  if  the  tour  arrives  a  6^  without  elimin- 
ating X  ,  then  a  parametric  segment  from  g   to  (3-d)  may  be  inserted  into 
the  tour  and  test  (6.7)  performed.   Figure  4  illustrates  how  the  path 

shown  in  Figure  2  might  be  altered  by  the  introduction  of  daO.   It  is 

2  5 

again  supposed  that  x  is  eliminated  by  a  direct  hit  from  A  and  that  x 

is  eliminated  by  an  indirect  hit  from  B.   Our  computer  implementation  of 

Step  5  has  been  organized  in  this  fashion. 

The  modified  hybrid  algorithm  may  now  be  summarized  as  follows. 

We  shall  sidestep  the  question  of  what  constitutes  an  "approximate" 

solution  of  (6.1)  by  assuming  that  an  optimal  solution  must  be  found  for 

each  ee[0,l]. 


Modified  Hybrid  Algorithm 

Step  1.   Set  n=l,  X  =  S  ,  and  choose  L>1.   Use  the  heuristic  to  construct 

an  initial  LB(e)  for  0<e^l. 

Step  2.   Construct  X  by  eliminating  all  elements  of  X  that  are  infeasible 
n  n 

for  every  O^GSl. 

e  f 

Step  3.   Construct  X  by  eliminating  all  dominated  elements  of  X  . 

Step  4.   If  Ix^l  ^  L,  set  X^  =  X^  and  go  to  Step  9. 
'  n'  n    n     "        ^ 

Step  5.   Use  linear  programming  to  do  elimination  by  bound  according  to  the 
tests  (6.6)  and  (6.7).   Let  X  denote  the  set  of  survivors. 
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Figure  4.   A  typical  resource-space  tour  in  R  when  d>0. 


Step  6.   (omit)  ~^^~ 

Step  7.   Apply  the  heuristic  to  each  of  the  surviving  residual  problems 
for  several  values  of  6.   Use  the  results  to  improve  LB(e). 
Save  all  incxmbents. 

Step  8.   (omit) 

Step  9.   If  n=N,  stop:  use  the  elements  of  X^  to  improve  LB(e);  then 

LB(e)=g(e)  for  0<e<l  and  each  incumbent  is  optimal  for  its  interval 

0    s 
of  e.   Otherwise  set  n=n+l,  generate  X  =  X^_^  x  S^,  and  go  to  Step  2. 

The  modified  hybrid  algorithm  was  tested  on  Problems  1,  4,  and  6  from 
Section  5.   The  results  are  summarized  in  Tables  5,  6,  and  7  respectively. 
In  each  run  the  direction  vector  d  was  taken  as  some  percentage  of  the 
initial  right-hand-side  b.   For  example,  if  d=5%  b,  then  (6.1)  has  right- 
hand-sides  b+e(.05)  b,  for  O<0<1.   A  0%  case  from  Section  5  is  included  for 
comparison.   The  "Solutions"  row  gives  the  number  of  optimal  solutions  found, 
or  equivalently  the  number  of  steps  in  the  g(0)  function.   The  "H  applied" 
row  gives  the  number  of  (evenly  spaced)  9  values  at  which  the  heuristic 
is  applied  at  Step  7.   (Petersen  was  used  at  Step  1  and  Toyoda  at  Step  7.) 
The  other  rows  are  labelled  as  in  Section  5.   Successful  applications  of 
the  (6.7)  test  are  counted  among  the  direct  hits  and  also  listed  separately 
as  "(6.7)  hits".   A  threshold  of  L=100  was  used  for  all  runs.   The  computation 
times  seem  quite  large  in  terms  of  the  length  of  the  interval  of  parameter- 
ization (5,  10,  or  15%).   They  are  not  unreasonable,  however,  in  terms  of 
the  number  of  optimal  solutions  found.   In  all  cases  the  computation  time 
is  considerably  less  than  the  number  of  optimal  solutions  found  multiplied 
by  the  computation  time  for  finding  npo  r.TM-iTt\«l  solution  in  the  0%  case. 

Although  developed  in  the  special  context  of  the  hybrid  DP/B&B 
approach,  the  new  bounding  test  (6.5)  is  of  much  wider  applicability. 
Through  its  use  any  LP-based  branch-and-bound  algorithm  can  be  converted 
to  an  algorithm  for  parametric  integer  programming.   The  extension  of  our 
results  to  the  general  branch-and-bound  context  is  the  subject  of  a 


TABLE  5.  PARAMETRIC  RESULTS  FOR  PROBLEM  1   (5x30) 


0% 

5% 

10% 

15% 

Solutions 

1 

11 

28 

37 

H  appl  ied 

1 

10 

20 

20 

LP  &  H  times 

1 

4 

6 

9 

Eliminations: 

Feasibility 

53 

190 

163 

144 

Dominance 

58 

125 

211 

372 

r(K),   (3.1) 

187 

165 

140 

122 

BRR,    (3.2) 

29 

- 

- 

- 

Direct  hits 

17 

38 

47 

51 

(6.7)  hits 

- 

3 

9 

16 

Indirect  hits 

79 

380 

468 

628 

Total  LP  pivots 

153 

529 

1173 

2606 

Time  (sec.) 

1.605 

8.114 

18.005 

43.304 

TABLE  6.  PARAMETRIC  RESULTS  FOR  PROBLEM  4  (10x28) 


0% 

5% 

10% 

15% 

Solutions 

1 

16 

29 

42 

H  applied 

1 

10 

20 

20 

LP  &  H  times 

1 

3 

4 

6 

Eliminations: 

Feasibility 

14 

19 

15 

13 

Dominance 

95 

316 

365 

437 

r(K).   (3.1) 

189 

170 

146 

140 

BRR,   (3.2) 

70 

- 

- 

- 

Direct  hits 

7 

19 

63 

100 

(6.7)  hits 

- 

11 

53 

88 

Indirect  hits 

86 

252 

278 

305 

Total   LP  pivots 

66 

173 

645 

1621 

Time  (sec.) 

1.155 

4.469 

13.129 

30.888 

TABLE  7.  PARAMETRIC  RESULTS  FOR  PROBLEM  6  (20x30) 


0% 


272% 


5% 


Solutions 

1 

6 

12 

H  applied 

1 

5 

10 

LP  &  H  times 

2 

4 

7 

Eliminations: 

Feasibility 

47 

122 

137 

Dominance 

0 

1 

12 

r(K).   (3.1) 

104 

90 

77 

BRR,   (3.2) 

57 

- 

- 

Direct  hits 

16 

31 

52 

(6.7)  hits 

- 

1 

21 

Indirect  hits 

222 

438 

682 

Total  LP  pivots 

180 

400 

1350 

Time  (sec. ) 

3.242 

9.486 

32.185 
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7.   Dropping  the  Non-Negativity  Assumption 

Allowing  negative  objective  function  values  does  not  require  any 

change  to  the  hybrid  algorithm.   Allowing  negative  values  in  the  constraint 

functions  and  on  the  right-hand-side  requires  the  feasibility  test  at  Step  2 

to  be  modified  or  abandoned.   The  non-negativity  assumption  insures  that 

every  descendant  of  the  partial  solution  xeX  will  consume  at  least  as 
^  n 

much  of  each  resource  as  x  does.   This  implies  that  x  has  no  feasible 
descendants  whenever  3.  >  b,  for  some  i.  Without  this  assumption  we  must 
either  abandon  the  feasibility  test,  in  which  case  we  may  redefine 

X  =  {xeX   I  X  is  efficient  with  respect  to  X  } 
n       n  '  n 

at  Step  3,  or  else  replace  it  with  some  weaker  sufficient  condition  for 
eliminating  x.   For  example:   x  has  no  feasible  descendants  if  for  some  i, 
3 .  >  b  .  and 

1      X 

N 

B^  +  y     min  {0,  min  {a..(k)  I  k=l K.}}>  b^ . 

^   j=n+l  ij     I     •     ♦   J     i 

Even  when  Step  2  is  omitted,  some  elimination  by  reason  of  infeasibility 
takes  place  as  a  special  case  of  elimination  by  bound.   If  xeX  does  not 
have  any  feasible  descendants,  then  its  residual  problem  (2.2)  is  infeasible 
and  this  may  be  detected  when  the  LP  relaxation  (3.3)  is  solved.  When 
this  happens  x  is  eliminated  by  bound,  provided  we  adopt  the  usual  convention 
that  the  optimal  value  of  an  infeasible  maximization  problem  is  (  -  ") . 

The  resource  space  tour  must  be  modified  to  share  extreme  rays  as  well 

e       el       Q 
as  extreme  points  among  the  members  of  X  .   Let  X  =  {x  ,  ...,  x  }  and 

consider  the  linear  programs  (3.3)  for  6  =  B  ,  ...,  B^.   If  one  of  these  is 

infeasible,  then  the  corresponding  dual  (3. A)  has  an  unbounded  solution  along 
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an  extreme   ray    (w,    z)    of  D    , , .      As   soon  as   this   extreme   ray  is   obtained  it 
can  be  used  to  perform  what   amounts    to  a   feasibility   test   on  each  x    ,      That 
is,   X     has  no   feasible  descendants   if 


N 


y       w.    (b.-B?)   +  y  z.K. 

1=1  1=n+l     -^   -^ 


.    <   0 


since  this  condition  means  that  UB  , ^ (b-g^)  =  -  ». 

n+1 

The  most  important  consequence  of  dropping  the  non-negativity 
assumption  is  that  it  becomes  much  more  difficult  to  devise  good  heuristics. 
Intuitive  or  common  sense  approaches  to  "knapsack  type"  problems  break  down 
when  negative  data  is  admitted,  and  there  is  no  easy  way  to  round  an  LP 
solution  and  obtain  a  feasible  integer  solution.   Heuristics  for  general 
integer  programs,  such  as  those  of  Hillier  [13]  and  Kochenberger  et.  al. 
[16 ] ,  would  have  to  be  incorporated  into  the  hybrid  algorithm. 
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8.   Conclusion 


This  paper  has  presented  a  hybrid  DP/B&B  algorithm  for  separable 
discrete  mathematical  programs  and  evidence  of  its  computational  viability. 
If  the  hybrid  algorithm  is  viewed  as  dynamic  programming,  then  the  intro- 
duction of  bounding  arguments  serves  to  reduce  the  size  of  the  state  space 
at  each  stage  and  enables  us  to  compute  an  optimal  solution  for  one  particular 
right-hand-side  vector,  b,  rather  than  for  all  0<b'<b.   If  on  the  other  hand 
the  hybrid  algorithm  is  viewed  as  b ranch- and-bound,  then  the  incorporation 
of  a  DP  framework  has  two  main  consequences.   First,  DP  provides  an  addition- 
al fathoming  technique:   dominance.   Second,  and  of  greater  importance,  DP 
takes  control  of  the  search  strategy.   The  B&B  methodology  achieves  its 
great  flexibility  by  leaving  its  user  with  many  different  choices  to  make. 
Among  these  are:   how  to  separate  a  node  that  cannot  be  fathomed  (e.g. 
which  variable  to  branch  on)  and  which  node  to  attempt  to  fathom  next. (e.g. 
depth  first,  breadth  first,  best  bound,  priority,  etc).   In  the  hybrid 
algorithm,  the  DP  framework  dictates  that  the  same  branching  variable  be 
used  across  each  level  of  the  search  tree  and  that  we  attempt  to  fathom  all 
of  the  nodes  at  the  current  level  of  the  tree  before  proceeding  to  the  next 
level.   The  only  freedom  left  is  in  the  choice  of  which  variable  to  associate 
with  each  level  of  the  tree  and  in  what  order  to  consider  the  nodes  at  the 
current  level.   This  rather  rigid  structure  leads  directly  to  the  surprisingly 
effective  "resource-space  tour"  technique  for  computing  and  sharing  bounds. 

Our  ultimate  breadth  first  search  strategy  is  admittedly  an  extreme 
one.   It  is  quite  possible,  however,  for  a  more  conventional  branch-and- 
bound  procedure  to  use  the  hybrid  algorithm  to  fathom  particular  sub-trees 
while  retaining  higher-level  strategic  control.   We  have  not  yet  attempted 
this  but  it  appears  to  be  an  exciting  avenue  for  further  research. 
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At  the  conceptual  level,  the  central  role  of  the  optimal  return 
function  in  DP  has  lead  to  the  discovery  of  a  generalization  of  the  usual 
B&B  bounding  test  which  makes  it  possible  to  solve,  in  one  s.earch,  a  family 
of  parametric  integer  programs  whose  right-hand-sides  lie  on  a  given  line 
segment.   The  idea  may  be  readily  extended  to  families  whose  right-hand- 
sides  lie  in  more  general  sets,  for  example  a  hyper-rectangle  centered  at 
b  [19]. 

It  is  our  hope  that,  beyond  its  computational  value,  our  work  will 
have  further  theoretical  ramifications  and  will  lead  to  a  unifying  frame- 
work for  discrete  optimization.   That  is,  this  work  may  help  to  break  down 
the  artificial  barriers  which  exist  between  DP  and  B&B.   We  have  made  a 
start  in  this  direction  by  showing  how  bounding  arguments  may  be  used  to 
enhance  any  dynamic  programming  algorithm  [22],  not  just  the  special  one 
considered  here.   Furthermore,  we  feel  that  the  hybrid  viewpoint  will  lead 
to  a  deeper  understanding  of  right-hand-side  sensitivity.   In  view  of  the 
intimate  relationship  between  right-hand-side  sensitivity  and  duality  for 
convex  programs,  this  may  ultimately  result  in  new  concepts  of  duality  for 
discrete  programs. 
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